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ABSTRACT 


Driving is a very natural task to humans, yet a computer 
has difficulties not only to drive a car, but also to detect 
obstacles or to find a route to drive. Many have covered 
intelligent systems as a solution to the latter two 
problems, yet these systems require time to learn. Expert 
systems on the other hand, have a predefined collection 
of knowledge eliminating the necessity of learning, yet 
they can never cover all possible scenarios. This paper 
aims to show how algorithms can be a possible solution. 
They were used to detect obstacles and plan a route. To 
evaluate how well they performed, a robot was designed, 
which had to navigate itself through a building, using 
depth-maps and A* algorithms with corresponding 
hardware, to detect obstacles and calculate routes, 
respectively. 


1. INTRODUCTION 


This project aimed to create a plugin for any vehicle 
system, offering the benefits of obstacle detection using 
depth maps as well as route planning using A* 
algorithms. 


1.1. Depth from xBox Kinect 


Figure 1. Kinect infrared grid against wall 


A depth map is a two-dimensional matrix containing 
values about the depth of space from a particular 
perspective point. This depth map was supplied by the 
xBox Kinect. It uses infrared beams to project a grid of 
dots that contains area-identifying arrays of points 
(figure 1). These points or pixels can be read by a 


System on Chip (SoC) developed by Primesense [1]. The 
Kinect samples each of the 320 x 240 pixels depth value 
with 11-bit accuracy and sends it to OpenNI [2] drivers, 
interfacing the Kinect with a computer. The final 
application then receives a depth-map containing 
unsigned shorts or 16 bit numbers [3]. 


Using the depth-maps rather than camera images 
allows working with the actual depth of a room or the 
space in front of the lens rather than a two-dimensional 
representation. 


Next to the depth-mapping features, the Kinect has a 
built in motor and three-axis accelerometer, which were 
used to stabilise the Kinect’s orientation to always face 
horizontal and hence parallel to the ground. 


1.2. Route planning 


There are many ways to calculate a path from start to a 
finish position. The easiest approach is to simply start 
driving in the direction of the destination, until an 
obstacle is met. Then by using either the keep-left or 
keep-right rule one can navigate around the obstacle, yet 
this method is very slow, inefficient and may not always 
find a solution. 


Alternatively, the most common approach is to use a 
map on which it is tried to connect the known start and 
finish location with a line. Using algorithms and 
weighted routes, where the weight could describe the 
speed limit or average congestion of the road, navigation 
systems find the best path connecting start and finish. If 
the area’s or street’s properties are unknown, one has to 
rely on them all having equal properties. 


The A* (A-Star) algorithm is a perfect solution, since 
it is a simple algorithm and propagates a path over 
unknown, evenly weighted territory, using a map. This 
algorithm was implemented using C# and a blueprint of 
the testing location (School of systems engineering 
building on the University of Reading). 


2. OBSTACLE DETECTION 


This section covers how the xBox Kinect was used to 
allow a computer to detect obstacles of any shape or 
form in front of the device. It addresses the Kinect itself 
and how it was interfaced, what algorithm was used to 
translate and interpret the depth-map data and what 
results were achieved. 
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Figure 2. Test Application interfacing with depth-map (centre, bitmap used to represent data) and camera (right) 


2.1. Kinect device 


Two different drivers were implemented to interface 
with the Kinect. The non-profit organization OpenNI 
provided the required depth-map, video and audio 
drivers, whereas CL NUI drivers were used to interface 
with the Kinect’s motor, accelerometer and status LED 
[4]. Connected over USB to a computer, all the data was 
made available to the application. 


The depth-map, as well as the camera picture, was 
transmitted with a resolution of 640 x 480 pixels. Using 
pointer-interfacing commands, the map and _ picture 
could be read into memory and stored as a matrix and a 
bitmap respectively. The lenses on the cameras for 
picture and depth-map, widened the field of view of the 
Kinect to 58.1° (1.0144rad) horizontally and 45.3° 
(0.7898rad) vertically. 


2.2. Algorithm and settings 


Knowing that each entry in the depth matrix corresponds 
to exactly one distinct distance and direction, pointing to 
the closes recognised object, one can use this 
information in combination with simple trigonometry to 
calculate whether the given distance indicates an object 
or not. 
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Figure 3. Obstacle height detection 


A scanning-comparison algorithm using equation | 
then stored the forward field of view in a one- 
dimensional matrix. 


dyy sind = d,y sin(S-y<) = Hy (1) 


Equation | calculates the height of the ahead ground 
where, dx, is an entry from the depth matrix, a 
represents the vertical field of view of the Kinect, Y 


(upper case) represents the number of columns in the 
matrix (here 480) and y (Jower case) is the value of the 
column from which d,, was read. This is applied to 
every entry in the depth matrix and filtering the lowest 
value per column where: 


hy — Ny < Hq < he (2) 


Here hy, is the vehicle height, or the height of the lens 
above ground, h, is the maximum obstacle height that 
can be driven over and h, is the minimum ceiling height 
the car can drive under. For computational speed, this 
was implemented: 


lHal < hy ~ hy (3) 


The result is therefore a one-dimensional matrix 
containing the distance and direction to the closest 
obstacle. Since each entry in the [1, X] matrix has 
exactly one horizontal angle assigned, it directly relates 
to the area ahead of the Kinect. 


2.3. Results 


Corresponding to figure 2, where the depth-map is 
displayed using a bitmap, the resulting algorithm could 
detect the obstacle (box to the right) in under 0.1s. 
Figure 4 shows the resulting one-dimensional matrix, 
represented on a Im grid, where the further up the black 
bars start, the further away the obstacle-point is. As seen 
the box is clearly visible about the two meter mark on he 
camera picture as well as on the matrix representation. 


Figure 4. 1D matrix representation of closest obstacle 


The algorithm calculated reliably, yet the system’s 
only weakness was its vulnerability to tilting. This 


means when the Kinect was not leveled with the ground, 
its sensors were detecting ground as a close object and 
triggered this an obstacle. To overcome that problem, the 
Kinect’s accelerometer and built in motor were used to 
levels itself before setting off. 


3. ROUTE PLANNING 


This section covers the route planning part of the project. 
It includes which algorithm was picked, how the 
algorithm works, how it was implemented and how well 
it performed with different settings. 


3.1. Algorithm selection 


There are many algorithms that calculate paths. Many of 
them use predefined path weights, making faster routes 
more likely to be chosen. Others have areal weighting 
allocated, where a map is split into small, dot-like areas, 
which are individually assigned with a weight, dictating 
how drivable the terrain is. The path of shortest distance 
and highest drivability is then chosen by algorithm. 


When given a blueprint, apart from wall locations, 
there is no information about the terrain. Assuming that 
every floor is flat and equally drivable, one can use a 
simple next-neighbour exploring algorithm to find the 
shortest path between two points. The simplest is the A* 
algorithm. 


3.2. A* algorithm 


The algorithm uses nodes to establish the next best path. 
It is based upon a next-neighbour exploration and 
evaluation principle, where a node represents a point on 
a map [5]. That means the algorithm visits a node and 
evaluates the surrounding nodes on the following 
characteristics: 


- Distance to node from start (G) 
- Distance from node to the finish (H) 
-  Visiteability 
(in this case, whether the node is a wall or not) 
- Pointer to previous node 
(the previous node is the visited node that 
watched and changed this node) 
- Weight (F =G+H) 


These characteristics need to be found for each 
visited node. The rules for which node to visit next and 
when to update the surrounding nodes’ characteristics 
must follow these two rules: 


- The next visited node must be the one that has 
not already been visited and has the lowest 
weight allocated. 

- The surrounding nodes must only be updated if, 
from the currently visited node, the distance to 
the watched node (G) is shorter than the already 
allocated distance. 


To determine which nodes have been looked at or not, 
lists were used. One list contained all visited nodes. This 
list contained all possible nodes that could lie on the 


final route. Furthermore, it must never be empty, 
because that would suggest that no route could be found. 
The other list contained all watched nodes that have not 
yet been visited but watched. This list must always be 
sorted by weight (F) from lowest first, to highest last. 
This allowed to pick the top node on the watched list as 
the node to visit and evaluate next. Of course, when 
visited, the node must be added to the visited-list and 
removed from the watched-list. 


The algorithm hence ran though many possible nodes 
until the currently visited node has a remaining distance 
of zero and therefore lay on the finish location. To get 
the path, one simply had to backtrack all pointers and 
store them in a list. This path, as one may have already 
found, is backwards, leading from finish to start. By 
reversing the order, this was repaired. 


La 


Figure 5. Path calculation visualisation 


Neither calculating the weight nor extracting data 
from a bitmap is very computational intensive and could 
be optimized to a high degree. The issue with the A* 
algorithm was, that there is no good weight allocation 
system, resulting in the algorithm even considering 
nodes in opposite direction. Different ways of 
calculating G and H were considered and tested. For G: 


- Actual distance by continuously adding the 
distance from start, covering all nodes passed 
(common approach) 

-  Triangulating the distance to the visited node 
(Vx?+y’) and adding the distance to the watched 
node (tested and implemented for faster 
computation) 


H estimation approaches were: 


- “Manhattan” — Summation (x+y) 
- “Euclidean” — Triangulation (Vx? +y’) 
- “Chebyshev” — Maximum (biggest of x or y) 


The result, as seen in figure 5, where the Manhattan 
and common approach for G were used, needed quite a 
lot of memory (dark area on left map) to calculate a path 
(line on the right map). Furthermore this calculation took 
considerable amount of time, which was of interest if 
spontaneous route changes must be done to i.e. avoid 
unforeseen obstacles. 


To improve and straighten the path, the same 
algorithm was applied in reverse order between nodes 
along the path that are the furthest apart but in direct line 
of sight. Once the first line of sight was established and a 
path-segment was computed, the node found was then 
chosen to become the next starting point and the process 
was repeated until the original start was reached. This 
ensured that only a minimum number of nodes were 
selected as start and finish. That allowed the plotting of 
waypoints, since only at these nodes did the direction of 
the path change. With trigonometry, the distance and 
direction between the waypoints could be calculated. 
Therefore, the output reduced to vectors that point 
towards the next waypoint, which correlates much closer 
to reality. 


3.3. Results 


The algorithm’s performance, measured by the time it 
took to calculate the path, was recorded for all different 
G and H calculation methods. These results were based 
upon the same problem as seen in figure 5. 


Actual distance | Triangulation 
Manhattan 11663 ms 8778 ms 
Euclidean 38098 ms 17105 ms 
Chebyshev 43468 ms 14092 ms 


Figure 6. Times taken for each A* modification 


The distance of the resulting paths by each algorithm 
varied minutely and therefore was not suitable for 
comparison. Still, the times in figure 6 show that the 
Manhattan approach using triangulation to calculate the 
covered distance was the fastest out of all six 
possibilities. The resulting paths followed a very similar 
pattern and only differed neglectably. 


4. CONCLUSION 


The so-called “semi-autonomous remote surveillance 
vehicle” implemented the obstacle detection and route 
planning algorithm as previously explained. The choice 
was made to not use intelligent machines or equal 
learning systems, since they are more difficult to 
implement and, depending on the environment given, 
perform differently. Algorithms, although very slow, are 
designed to always find a solution, if there is one. They 
base their progress on assumptions, estimations and 
weights associated with their data. 


The Kinect was used to feed an image to the robot, 
which captures the scene in front of the lens. It also 
supplied a matrix containing a depth-map, which enabled 
the detection of obstacles. The key here was to use the 
three dimensional information passed by the Kinect in 
combination with its lens and camera characteristics. 
This, with trigonometry, allowed calculating the height 


of ground ahead, which, if bigger than anything the 
vehicle could drive over, was found as an obstacle. 


The only downside of this method was its extreme 
vulnerability to tilt. The vertical tilt, about the left-right 
axis, was compensated for using the Kinect’s built in 
motor and accelerometer, but the tilt about the front-rear 
axis was problematic. This problem was only reduced by 
ensuring that the Kinect was as leveled as possible. 


Using the A* algorithm to calculate a route for the 
vehicle to travel is a quick and easy solution. Although 
this algorithm can take a lot of time for very complex 
problems or when big blueprints are used, it always finds 
an answer. The result was modified and improved to be 
more useable for the vehicle and to better correlate with 
the real world. Passing vectors between waypoints, 
which are in direct line of sight and lie on the calculated 
path, achieved this. 


This system was based upon blueprints and therefore 
was optimal for route planning over flat terrain. Passing 
a street map to the system would have resulted in equal 
success. Compensation for obstacles could not be 
achieved, since the terrain was unknown and had to be 
explored first. Still, the algorithm was a very reliable and 
robust one and worked brilliantly for the project. 
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